Web Caching and Zipf-like Distributions: Evidence and Implications

| This paper addresses two unresolved issues about web caching. The rst issue is whether web requests from a xed user community are distributed according to Zipf's law [22]. Several early studies have supported this claim [9], [5], [1] while other recent studies have suggested otherwise [16], [2]. The second issue relates to a number of recent studies on the characteristics of web proxy traces, which have shown that the hit-ratios and temporal locality of the traces exhibit certain asymptotic properties that are uniform across the di erent sets of the traces [4], [19], [7], [10], [15]. In particular, the question is whether these properties are inherent to web accesses or whether they are simply an artifact of the traces. An answer to these unresolved issues will facilitate both web cache resource planning and cache hierarchy design. We show that the answers to the two questions are related. We rst investigate the page request distribution seen by web proxy caches using traces from a variety of sources. We nd that the distribution does not follow Zipf's law precisely, but instead follows a Zipf-like distribution with the exponent varying from trace to trace. Furthermore, we nd that there is only (i) a weak correlation between the access frequency of a web page and its size and (ii) a weak correlation between access frequency and its rate of change. We then consider a simple model where the web accesses are independent and the reference probability of the documents follows a Zipf-like distribution. We nd that the model yields asymptotic behaviors that are consistent with the experimental observations, suggesting that the various observed properties of hit-ratios and temporal locality are indeed inherent to web accesses observed by proxies. Finally, we revisit web cache replacement algorithms and show that the algorithm that is suggested by this simple model performs best on real trace data. The results indicate that while page requests do indeed reveal short-term correlations and other structures, a simple model for an independent request stream following a Zipf-like distribution is su cient to capture certain asymptotic properties observed at web proxies. Keywords|caching, World Wide Web, Zipf distribution.